Filed under: Java, Public speaking, Selenium, Technical debt, Test automation, — Tags: 4 rules of simple design, Clean code — Thomas Sundberg — 2017-09-28
To support ever shorter release cycles you need to automate testing, and to do that, you need to use a program that can verify your desired behavior. One option is to use tools that can record and replay a scenario, but those are a nightmare to maintain, and you'll probably end up writing the code needed for automating the tests yourself.
For automating the testing of a web application, Selenium is a better way. It takes some programming skills, however, and you must take care to ensure that your tests are easy to understand and maintainable.
Writing maintainable, easy to understand tests might seem hard, but not if you follow these four, simple rules of design.
(more...)Filed under: TDD, Test automation, — Tags: BDD, JUnit, Java, Mockito, Testing, Unit tests — Thomas Sundberg — 2016-10-03
To gain confidence when testing software, you want to test the program as much as possible. The conventional way to do this is to test the application extensively through its external endpoints. These external endpoints may be a user interface or web services. They can almost always be be automated and automation is a good start.
Unfortunately, testing from the external endpoints leads to a few problems:
The cure is to rely as much as you can on fast unit tests. But a unit test will only test one thing. To know if a class can collaborate with other classes, you need to test that collaboration scenario. This can lead to integrated tests that have bad diagnosis precision, are slow, and have too many execution paths.
There is one alternative, though, that many developers hasn't explored enough. That is using unit tests with mocks and stubs in a strict way. I will explore this alternative in this post.
(more...)Filed under: Automation, Quality, Technical debt, Test automation, — Tags: Stress reduction — Thomas Sundberg — 2016-09-01
Brian McCallister tweeted "Current development speed is a function of past development quality."
It kind of summaries why I am passionate about code quality and taking the time early to do the right thing right.
(more...)Filed under: Load testing, Scala, Test automation, Tools, — Tags: Gatling — Thomas Sundberg — 2016-08-04
The documentation for Gatling tells us that "Gatling is an open-source load testing framework based on Scala, Akka and Netty". I have heard good things about it but never tried it.
The technical parts, Scala, Akka and Netty, were less interesting from my perspective. The interesting part was its developer-friendly DSL, Domain Specific Language. I have used tools like JMeter and I have never been impressed by their user interfaces. They are too complicated. As a developer, I like when I can specify my load script using my regular tooling. That is, an editor, a build tool and a command line.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: Behaviour-Driven Development, Testing iceberg — Thomas Sundberg — 2016-07-25
No tool is perfect for all jobs. When you start out automating Behvaiour-Driven Development, BDD, using Cucumber you tend to use it for much more than you need. This will work, but it is not a good idea for a few reasons:
Filed under: TDD, Test automation, — Tags: JUnit, Java — Thomas Sundberg — 2016-03-25
A test must be independent. That is, a test should never depend on the result of another test.
If you run two tests in a specific order, you should expect the same result as if you run them in the reverse order.
(more...)Filed under: Java, Selenium, Test automation, — Tags: Page Object Pattern, WebDriver — Thomas Sundberg — 2016-01-14
A lot of people want to automate testing of their web applications. This is definitely a good thing. But it happens that they focus more on the tooling than the testing.
(more...)Filed under: Gradle, Java, Programming, Test automation, Tools, — Tags: Acceptance Test Driven Development, Acceptance testing, Acceptance tests, Automation, Separating tests — Thomas Sundberg — 2015-04-29
It is very convenient to run the unit tests separated from other, slower, tests. There are different ways to do this. One way is to have a separate module for the acceptance tests.
Separating the modules is acceptable in some cases. It is not acceptable in others. There is a simple way to separate the source code for the acceptance tests while keeping the it in the same project if you use Gradle. Separate the tests with different source sets.
Separation on source sets means that you will keep all the acceptance tests in another directory structure than the unit tests. If you use the usual separation of production code and test code, then all you want to do is to add a new source set that contains the acceptance tests.
(more...)Filed under: Cucumber, Requirements, Software development, Test automation, — Tags: BDD, Behaviour Driven Development, Behaviour Driven Development - BDD, Cucumber, Cucumber-jvm, Executable specifications, I T.A.K.E. Unconference 2014, Java, Living documentation, Maven, Test automation — Thomas Sundberg — 2014-05-29
This is the example I showed at the I T.A.K.E. Unconference 2014 in Bucharest. I created it for your convenience so you should be able to implement it yourself after the presentation.
Before we dive into the example, let me just recap what I am aiming for. I will show you how an example (or specification if you want) can be executed. The example is written in plain text and it is used as the basis for an execution. This example can later be relied upon for regression testing as well as living documentation.
(more...)Filed under: Java, TDD, Test automation, — Tags: JUnit, JUnitParams — Thomas Sundberg — 2014-04-24
How can you pass a parameter to a unit test parametrised with JUnitParams? The answer is simple: As any other parameter.
(more...)Filed under: Cucumber, Executable specification, Test automation, — Tags: Automated acceptance criteria, Automated acceptance tests, Collaboration, Concrete examples, Cucumber-jvm, JUnit, Readability, Requirements — Thomas Sundberg — 2014-03-30
Before I can try to motivate why you should use a tool, let me define what it is and what it does.
Cucumber is a tool for collaboration and testing. It is used to create examples of behaviour that are executable. Creating examples in a collaborative way emphasize close cooperation between business analysts, testers and developers. The examples they come up with can be used as acceptance tests for the system being developed. It can be used as a testing tool where the tests are defined in a business friendly language while still being executable.
Our goal is to create a common understanding of the problem and therefore simplify the communication between all parties involved. We would also like to get something that is possible to use for automating the verification of the resulting program. That is, use as a base for test automation of the system.
(more...)Filed under: Software development, Test automation, — Tags: Why test — Thomas Sundberg — 2014-01-28
A lot of people and companies are talking about and thinking of test automation. It seems to be like a holy grail within software development for a lot of people.
I am, however, sometimes wondering if they have thought this through properly. A few questions are important to know the answer to before you start a project that aims to place a product, or a project if you want, under automated testing. It seems to me as if a lot of people haven’t thought enough of the question why or the question what. Focus is often on the question how and almost never on where or when. How is obviously important, but if why and what hasn’t been properly understood then the how is uninteresting. Where is also important to decide upon. Some things should be tested through the final user interface, some things should be tested on the inside of a system. Some things should be tested with unit tests and some things at other levels. Finally, it is important to decide when the tests should be implemented. Tests can be implemented first, last or during the implementation of the system. This may not seem to a big deal, but it turns out that it is.
But let us start with the why.
(more...)Filed under: Test automation, — Tags: Dependency injection, Test Driven Development — Thomas Sundberg — 2013-01-31
Testing is all about verifying that something works as expected. What are the challenges then? There are many, one very obvious is to actually know how it should work. Another challenge is to isolate the system under test so that it is possible to know what is tested and determine if that specific part works as expected.
(more...)Filed under: JUnit, Java, Test automation, — Tags: Bad tests, Cobertura, False positives, Good tests, No assert, Test coverage, Unit tests — Thomas Sundberg — 2012-12-18
Measuring the test coverage is something many people think is a good idea. But is it a good idea? It depends on the quality of the tests. Test coverage may be a good measurement if the tests are good. But suppose that we have a high degree of test coverage and bad tests?
I will show two examples of how to get a 100% test coverage using Cobertura. They will be based on a set of good tests and a set of bad tests.
Test coverage is calculated by recording which lines of code that has been executed. If the code has been executed through a test in a testing framework then we have calculated the test coverage.
The calculation is done using these steps:
We will be able to say that 47% of the lines in the source code has been executed. If the execution is done through test code, this will give us a measurement of the test coverage.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Cucumber is a tool that supports Behaviour Driven Development, BDD. A lot of people think that the only place where a system has behaviour is in the user interface and especially in the graphical user interface. As a developer I know that this is not the case. All systems have behaviour at different places and different levels.
I will show an example of how a system can be developed using its desired behaviour and start from a non graphical point. I will work from the model down to the database and when I'm happy with the logical behaviour I will add a graphical user interface on top of it. I will actually add a few different interfaces; two web-based, one swing and two different types of web services. The result will be an example of Model View Controller, MVC, developed using BDD.
An important point when I add the GUIs or web services is that I will not change the desired behaviour. I will only change how the behaviour is verified. This is one way of showing you that Cucumber and BDD is not about testing GUIs. It is about systems behaviour.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
The feature I will start with looks like this:
src/test/resources/se/waymark/rentit/Rent.feature
Feature: Rental cars should be possible to rent to gain revenue to the rental company.
As an owner of a car rental company
I want to make cars available for renting
So I can make money
Scenario: Find and rent a car
Given there are 18 cars available for rental
When I rent one
Then there will only be 17 cars available for rental
It consists of three parts:
Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Many modern applications are built as web applications. The benefits are obvious, you don't need to package your software in shrink-wrap and send it to your customers. Upgrading is easy, you have to upgrade the server you host the system on and that's it.
The first user interface I will add to the rental system will therefore be a web GUI. It will be the simplest possible solution and the goal is not to build a fancy web app. The goal is to show how Cucumber can control a tool like Selenium WebDriver to assert the behaviour of the web application.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Previous - A JSF web application
A wicket application is yet another web application. I divide the project in two parts as earlier. The only large difference is the support class that will connect to the system under test. It has been adapted for another web application.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Previous - A Wicket web application
A Java Swing application is yet another graphical user interface that can be attached on top of the model developed earlier. The project is divided in the same way as earlier, in two parts. The only large difference here is the support class. It need to be adapted for a Swing user interface. Another difference is of obviously that the GUI is developed using Swing. But that has actually a rather small impact on the verification.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Previous - A Swing application
A RESTFul web service is yet another way to use the model. It doesn't have any user interface and it is expected to be used by third party suppliers. I divide the project using two Maven modules as earlier. One for the production code and one for the verification. The only large difference here is the support class that will connect to the system under test but now has to be adapted for a RESTFul web service instead of any graphical interface. Another difference is of course that there is no GUI and that it is developed using a Jersey servlet instead, but that has little impact on the verification.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
Previous - A RESTFul web service
A SOAP Web Service is yet another way to use the model. It doesn't have any user interface and is expected to be used by third party suppliers. I divide the project using two Maven modules as earlier. One for the production code and one for the verification. The only large difference here is the support class that will connect to the system under test but now has to be adapted for a SOAP web service instead of any graphical interface or RESTFul web service. Another difference is of course that there is no GUI and that it is developed using a CXF instead, but that has little impact on the verification.
(more...)Filed under: BDD, Cucumber, Test automation, — Tags: CXF, Cucumber-jvm, JSF, JUnit, Jersey, MVC, Model view controller, REST, RESTAssured, RESTFul, Selenium, Soap, Swing,, Swinggui, WebDriver, Wicket — Thomas Sundberg — 2012-11-01
I have shown you five different ways to implement something on top of a common model using the same behaviour. I hope you can agree that it is obvious that BDD and Cucumber-JVM is not just meant for testing GUIs. It can be used to assert any behaviour.
(more...)Filed under: Java, Maven, Selenium, Test automation, — Tags: WebDriver — Thomas Sundberg — 2012-10-29
Getting started with Selenium WebDriver may be an issue. You must write some code and get the code running. I have created what I think is the smallest possible solution that could work. It consists of two files, a project definition and the actual test.
You will need to have Java and Maven installed. I will not tell you how this should be done, it depends on your environment and operating system.
(more...)Filed under: Cucumber, Java, Test automation, — Tags: Acceptance test, Acceptance test driven development, Agile Cambridge 2012, Cucumber-jvm, Specifications by example — Thomas Sundberg — 2012-09-28
An example is perhaps the best way to describe something. Concrete examples are easier to understand than abstract descriptions.
I will show how Cucumber-JVM can be used to specify an example and how the example can be connected to the system under test, SUT. The example can then be executed using any IDE or using Maven.
(more...)Filed under: Java, Maven, TDD, Test automation, — Tags: Acceptance test, Integration test, Separation of concern, System tests, Test suit — Thomas Sundberg — 2012-08-21
The way Maven file structure is defined is a great way to separate unit tests and production code. Unit tests are fast. So fast that developers developing using Test Driven Development, TDD, doesn't have any performance problems when applying the Red-Green-Refactor cycle in their work. A large unit test suit is expected to be executed in seconds.
There is, however, no standard on how the files should be structured for slower tests (integration/acceptance/system et.al.). A common property for these tests is that they are slow. They often require a complete system setup in a known state. They use the filesystem, network, database and similar slow resources. Slow tests are normally not executed often. Developers seldom have the patience to wait for a build a long time before writing the next thing. Using them in a Red-Green-Refactor cycle is not practical. It takes too long time.
So what options do we have to separate the fast units tests and the slow tests? There are two main tracks that I have explored.
Filed under: Java, Selenium, TDD, Test automation, — Tags: @ClassRule, @Rule, JUnit, Screen shot on failure — Thomas Sundberg — 2012-07-08
JUnit supports annotations so a method can be executed first in a test class or before each test method is executed. It also has annotations that supports methods to be executed after each test method or after the test class. This is very good if you need a common setup and a common tear down.
The after methods don't give you access to the test result. You cannot know if a test method failed or not. This may
pose a problem if you have a need to perform some specific actions after a failed test. A solution could be to
implement an onError() method. But JUnit doesn't support it.
A solution is to employ a @Rule annotation.
(more...)Filed under: Java, Maven, TDD, Test automation, — Tags: JUnit, Random, Surefire, runOrder — Thomas Sundberg — 2012-05-03
It works on my machine!
Ever heard that from a developer? I have, and it happens that the reason is that it actually works on their machine. It may also be the case that the order in which tests are executed matters. Test classes depends on each other and the order they are executed in is important. The solution is to execute the tests in random order so that any dependencies between tests are found and can be removed.
(more...)Filed under: Clean code, Java, TDD, Test automation, — Tags: Bad tests, Dependency injection, Good test, JUnit, JUnit, Mockito, Parameterized JUnit, Readability, Test harness — Thomas Sundberg — 2012-03-08
A colleague asked that question the other day. What is a good test? It is a really difficult question to answer. There are a number of properties that hold true for a good test. I will list some of them and discuss why I think they are important. Others may think that the order of importance is different or that other properties are more important then those that I list.
I will show some examples using Java and JUnit of what I think are bad tests and how they can be refactored to something better.
Tests are automated in my world. I always try to avoid manually testing. The main reason for this is that manual testing is slow and hard to repeat arbitrary many times. Therefore, test should be read as automated test in for the rest of this text.
(more...)Filed under: Java, Maven, Test automation, — Tags: Ant, Cobertura, Code coverage, JUnit, Test coverage — Thomas Sundberg — 2012-02-18
Test driving a project should result in a test coverage of 100% of the code. But how can we know what test coverage we have? Is it even important?
(more...)Filed under: Cucumber, Maven, Test automation, — Tags: BDD, Behaviour Driven Development, Behaviour Driven Development - BDD, Cucumber, Cucumber-jvm, Executable specifications, Java, POJO, Readability, SoapUI, Test web services, soap — Thomas Sundberg — 2011-11-16
An example is perhaps the best way to describe something. Concrete examples are easier to understand than abstract descriptions.
I will show how SoapUI can be used to test a web service. I will also show three different tools that can be used to control SoapUI. This tool chain can easily be made a part of your continuous build.
(more...)Filed under: Automation, Cucumber, Maven, Selenium, Software development, Test automation, — Tags: BDD, Behaviour Driven Development, Behaviour Driven Development - BDD, Cucumber, Cucumber-jvm, Executable specifications, Java, Page Object, Parameterized JUnit, WebDriver — Thomas Sundberg — 2011-10-18
Selenium a great tool for testing web applications. The current version, Selenium 2, is a merge between Selenium and WebDriver. I will walk you through an example where we test a web site using Selenium in a few different ways. This is the same example as I demonstrated at Scandev on tour in Stockholm 18 October 2011.
(more...)Filed under: Java, TDD, Test automation, — Tags: Bottom up, FEST Assert, Fluent assertions, Hamcrest, Hardcoded database, In memory database, Top down, assertThat — Thomas Sundberg — 2011-04-24
Hamcrest is a great framework for assertThat and it is bundled with JUnit. It is getting some competition from another framework, the FEST assertThat framework. The idea behind FEST is to use a fluent interface. This means that you can use your development environments code completion features to build your asserts. Its main goal is to improve test code readability and make maintenance of tests easier.
(more...)Filed under: Java, Selenium, Test automation, — Tags: Cargo, Integration test, JUnit, Jetty, Maven, Spring, TDD, Web application — Thomas Sundberg — 2009-04-17
We want to build a web application and we want to test it automatically.
(more...)
RSS